Admin Configuration

NF-Agent is a data collection engine with real-time pipelining capabilities. NF-Agent can dynamically unify data from disparate sources and normalize the data into destinations of your choice. Cleanse and democratize all your data for diverse advanced downstream analytics and visualization use cases.

 

While NF-Agent originally drove innovation in log collection, its capabilities extend well beyond that use case. Any type of event can be enriched and transformed with a broad array of log-source, filter, and log-server, with many native codecs further simplifying the ingestion process. NF-Agent accelerates your insights by harnessing a greater volume and variety of data.

Configuration UI allow admin to update the Log-Source and Log-Server configuration file in NF-Agent directly. Admin can update the plugin, path, type, technology, parser in the log-source configuration. Admin can also update Index prefix and output host in the log-server configuration.

Earlier, to create and change in the NF-Agent configuration files, admin has to do it from the back-end. Now, by adding Admin-configuration UI, admin can do this from the UI itself. No need to go to back-end to create and change in the configuration files.

Create Index

To do indexes management from configuration UI, follow the below mentioned steps:

  • Click the Configuration menu from the dropdown of configuration panel.
  • Then, click on Admin Configuration from drop down list.
Figure 292: Admin Configuration
  • This displays a list of indexes created.
Figure 293: List of Created Index
  • To create a new index, provide the Tier name and Env name in the box and click Add This creates an index with the name “env and tier” then added in to the table.

Note: User can delete an index by clicking the Delete icon

  • Now, user can add configuration for the created index. For this, first click over the index or row of the table. Then, on the right side, a section is displayed where user can add the Log-Source file and Log-Server files.
Figure 294: Log Source and Log Server

Log-Source File

Within the Log-Source tab, click the Add button. This displays a section where user needs to provide the following inputs:

  1. Plugin (either file or JournalD)
  2. Path of the log-source file from where the logs are to be captured (for example:/tmp/access*.log*)
  3. Type of log to be captured (for example: accesslog)
  4. Technology of the log-source file to be provide (for example: Netstrom_Trancactionlog_Parser)
  5. Add parser Based on the Technology selected (for example: filter-38).
Figure 295: Log Source

ADVANCE: If user wants to provide since db and start position click on Advance button to show.

6. Sincedb: Path of the sincedb database file (keeps track of the current position of monitored log files) that will be written to disk. The default value for since db is ‘/tmp’.

NOTE: It must be a file path and not a directory path.

7. Start Position: Choose where nfagent starts initially reading files: at the beginning or at the end. The default behavior treats files like live streams and thus starts at the end. If you have old data you want to import, set this to beginning.

 

Figure 296: Start Position
  • User can add multiple blocks and provide the above details for various other logs to be captured by clicking Add button.
  • Once all the details for log-source files are mentioned, click the Next button.

Log-Server File

  • To add the Log-Server configuration, click the Log-Server
  • For a new Log-server file, click the Add
  • A template is displayed to add the details. Select the index Prefix (For example: attr_prod) and output host (it should be in array with double quotes for example: [“10.10.40.16:9200”]).
  • Admin can also add multiple output hosts (For example: [“10.10.40.16:9200”,”10.10.40.19:9200”]).
Figure 297: Log Server

ADVANCE: If user wants to add manage_template then click on Advance  button to show.

Preview File

  • This section enables a user to see the preview of the various files, such as Log-Source, Log-server, filter that are configured / created for indexes management.
  • Here Parser from Log-Source are shown in different tab (for example: Filter-3, Filter-4).
  • If User wants to modify any file, then click on the Edit-Log-Source button of Input-Conf tab similar with other tabs i.e. Output-Conf and Filter-Conf.
  • Then click on Save button to save the modified data.
Figure 298: Preview

Log Parser Configuration

Log parser configuration involves setting up the log parser to extract the desired information from the log files. This typically involves defining rules or patterns that the parser will use to identify and extract the relevant data.

Figure 299: Log Parser Configuration

For configuration, we have to provide details in the following fields:

  • Tier name: Enter the tier name in this field.
  • Env name: Mention the environment of the tier.

Once fields are filled, click the add button.

Previous
Next